Table of Contents



InterRater Reliability (IRR)

IRR WHYTRANSP

Cohen’s Kappa and percentage of agreement across Coders 1 and 2 for each variable studied
Metric Cohen_Kappa Agreement_Percentage
VALID_TRANSP 0.83 96.03
ASPECTS_TRANSP 0.72 82.70
ABBR_TRANSP 0.86 99.48
PB_TRANSP 0.98 99.61
NTHEMES_TRANSP 0.37 59.63



IRR WHYTRUST

Cohen’s Kappa and percentage of agreement across Coders 1 and 2 for each variable studied
Metric Cohen_Kappa Agreement_Percentage
VALID_TRUST 0.53 87.32
ASPECTS_TRUST 0.77 88.66
ABBR_TRUST 0.66 98.91
PB_TRUST 0.42 90.16
NTHEMES_TRUST 0.30 47.68



Valid answers

Valid WHYTRANSP

Proportions per group WHYTRANSP

Proportions of non sense, not in line, DK and valid answers using the codes of Coder 1

Coder 1 Control Choice PushDictation PushRecording
1 Non sense 0.8 1.8 3.2 1.5
2 Not in line 7.4 5.1 3.6 5.5
3 DK 9.0 2.8 7.3 3.5
4 Valid 82.8 90.3 85.9 89.6
Number of observations 244 217 220 201
Proportions of non sense, not in line, DK and valid answers using the codes of Coder 2

Coder 2 Control Choice PushDictation PushRecording
1 Non sense 0.8 1.4 1.8 2.0
2 Not in line 4.9 4.6 5.5 4.0
3 DK 10.2 2.3 8.2 3.5
4 Valid 84.0 89.4 83.6 90.5
9 To be defined NA 2.3 0.9 NA
Number of observations 244 217 220 201
Proportions of non sense, not in line, DK and valid answers using the codes of Coder 3

Coder 3 Control Choice PushDictation PushRecording
1 Non sense 0.8 1.4 2.7 2.0
2 Not in line 7.8 5.5 5.0 5.0
3 DK 9.4 2.8 7.3 3.5
4 Valid 82.0 90.3 85.0 89.6
Number of observations 244 217 220 201

Tests of significance across groups WHYTRANSP

Tests non sense Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.132 ns 0.794 ns
Control PushRecording 0.827 ns 1.000 ns
PushDictation PushRecording 0.414 ns 1.000 ns
Control Choice 0.578 ns 1.000 ns
PushDictation Choice 0.557 ns 1.000 ns
PushRecording Choice 1.000 ns 1.000 ns
Tests non sense Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.590 ns 1 ns
Control PushRecording 0.514 ns 1 ns
PushDictation PushRecording 1.000 ns 1 ns
Control Choice 0.895 ns 1 ns
PushDictation Choice 1.000 ns 1 ns
PushRecording Choice 0.919 ns 1 ns
Tests non sense Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.223 ns 1 ns
Control PushRecording 0.514 ns 1 ns
PushDictation PushRecording 0.860 ns 1 ns
Control Choice 0.895 ns 1 ns
PushDictation Choice 0.514 ns 1 ns
PushRecording Choice 0.919 ns 1 ns
Tests not in line Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.122 ns 0.731 ns
Control PushRecording 0.537 ns 1.000 ns
PushDictation PushRecording 0.502 ns 1.000 ns
Control Choice 0.408 ns 1.000 ns
PushDictation Choice 0.617 ns 1.000 ns
PushRecording Choice 1.000 ns 1.000 ns
Tests not in line Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.960 ns 1 ns
Control PushRecording 0.806 ns 1 ns
PushDictation PushRecording 0.630 ns 1 ns
Control Choice 1.000 ns 1 ns
PushDictation Choice 0.853 ns 1 ns
PushRecording Choice 0.940 ns 1 ns
Tests not in line Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.303 ns 1 ns
Control PushRecording 0.316 ns 1 ns
PushDictation PushRecording 1.000 ns 1 ns
Control Choice 0.436 ns 1 ns
PushDictation Choice 0.973 ns 1 ns
PushRecording Choice 0.972 ns 1 ns
Tests DK Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.60700 ns 1.0000 ns
Control PushRecording 0.03070
0.1540 ns
PushDictation PushRecording 0.13500 ns 0.4050 ns
Control Choice 0.00906 ** 0.0544 ns
PushDictation Choice 0.05290 ns 0.2110 ns
PushRecording Choice 0.88800 ns 1.0000 ns
Tests DK Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.54500 ns 1.00000 ns
Control PushRecording 0.01030
0.05170 ns
PushDictation PushRecording 0.06700 ns 0.20100 ns
Control Choice 0.00111 ** 0.00665 **
PushDictation Choice 0.01120
0.05170 ns
PushRecording Choice 0.66900 ns 1.00000 ns
Tests DK Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.50500 ns 1.000 ns
Control PushRecording 0.02150
0.108 ns
PushDictation PushRecording 0.13500 ns 0.405 ns
Control Choice 0.00599 ** 0.036
PushDictation Choice 0.05290 ns 0.211 ns
PushRecording Choice 0.88800 ns 1.000 ns
Tests valid Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.4270 ns 0.972 ns
Control PushRecording 0.0573 ns 0.287 ns
PushDictation PushRecording 0.3240 ns 0.972 ns
Control Choice 0.0267
0.160 ns
PushDictation Choice 0.2020 ns 0.806 ns
PushRecording Choice 0.9210 ns 0.972 ns
Tests valid Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.0000 ns 1.000 ns
Control PushRecording 0.0581 ns 0.302 ns
PushDictation PushRecording 0.0504 ns 0.302 ns
Control Choice 0.1200 ns 0.418 ns
PushDictation Choice 0.1050 ns 0.418 ns
PushRecording Choice 0.8210 ns 1.000 ns
Tests valid Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.4520 ns 0.9040 ns
Control PushRecording 0.0340
0.1700 ns
PushDictation PushRecording 0.2120 ns 0.6350 ns
Control Choice 0.0147
0.0884 ns
PushDictation Choice 0.1220 ns 0.4890 ns
PushRecording Choice 0.9210 ns 0.9210 ns


Valid WHYTRUST

Proportions per group WHYTRUST

Proportions of non sense, not in line, DK and valid answers using the codes of Coder 1

Coder 1 Control Choice PushDictation PushRecording
1 Non sense 2.4 3.6 2.4 5.9
10 Depends 2.0 1.8 1.0 0.5
2 Not in line 3.7 4.9 3.8 2.5
3 DK 9.8 5.8 8.1 6.9
4 Valid 80.4 82.2 82.8 83.3
5 Refer previous answer 1.6 1.8 1.9 1.0
Number of observations 245 225 209 204
Proportions of non sense, not in line, DK and valid answers using the codes of Coder 2

Coder 2 Control Choice PushDictation PushRecording
1 Non sense 0.8 1.3 1.0 2.5
10 Depends 1.2 NA NA NA
2 Not in line 1.6 2.2 1.4 4.9
3 DK 6.1 1.8 4.3 2.9
4 Valid 86.9 89.3 89.5 88.2
9 To be defined 3.3 5.3 3.8 1.5
Number of observations 245 225 209 204
Proportions of non sense, not in line, DK and valid answers using the codes of Coder 3

Coder 3 Control Choice PushDictation PushRecording
1 Non sense 2.4 3.1 2.4 5.9
10 Depends 1.6 0.9 0.5 0.5
2 Not in line 3.7 3.6 3.3 2.0
3 DK 9.8 6.2 8.6 7.4
4 Valid 80.8 84.4 83.3 83.3
5 Refer previous answer 1.6 1.8 1.9 1.0
Number of observations 245 225 209 204

Tests of significance across groups WHYTRUST

Tests non sense Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1.000 ns
Control PushRecording 0.108 ns 0.651 ns
PushDictation PushRecording 0.124 ns 0.651 ns
Control Choice 0.665 ns 1.000 ns
PushDictation Choice 0.668 ns 1.000 ns
PushRecording Choice 0.362 ns 1.000 ns
Tests non sense Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1 ns
Control PushRecording 0.313 ns 1 ns
PushDictation PushRecording 0.427 ns 1 ns
Control Choice 0.924 ns 1 ns
PushDictation Choice 1.000 ns 1 ns
PushRecording Choice 0.619 ns 1 ns
Tests non sense Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1.000 ns
Control PushRecording 0.108 ns 0.651 ns
PushDictation PushRecording 0.124 ns 0.651 ns
Control Choice 0.876 ns 1.000 ns
PushDictation Choice 0.870 ns 1.000 ns
PushRecording Choice 0.247 ns 0.987 ns
Tests not in line Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1 ns
Control PushRecording 0.639 ns 1 ns
PushDictation PushRecording 0.604 ns 1 ns
Control Choice 0.672 ns 1 ns
PushDictation Choice 0.760 ns 1 ns
PushRecording Choice 0.282 ns 1 ns
Tests not in line Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.0000 ns 1.000 ns
Control PushRecording 0.0869 ns 0.496 ns
PushDictation PushRecording 0.0827 ns 0.496 ns
Control Choice 0.8970 ns 1.000 ns
PushDictation Choice 0.8010 ns 1.000 ns
PushRecording Choice 0.2130 ns 0.851 ns
Tests not in line Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1 ns
Control PushRecording 0.427 ns 1 ns
PushDictation PushRecording 0.568 ns 1 ns
Control Choice 1.000 ns 1 ns
PushDictation Choice 1.000 ns 1 ns
PushRecording Choice 0.479 ns 1 ns
Tests DK Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.652 ns 1.000 ns
Control PushRecording 0.346 ns 1.000 ns
PushDictation PushRecording 0.762 ns 1.000 ns
Control Choice 0.149 ns 0.892 ns
PushDictation Choice 0.437 ns 1.000 ns
PushRecording Choice 0.792 ns 1.000 ns
Tests DK Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.5150 ns 1.000 ns
Control PushRecording 0.1720 ns 0.861 ns
PushDictation PushRecording 0.6320 ns 1.000 ns
Control Choice 0.0312
0.187 ns
PushDictation Choice 0.2070 ns 0.861 ns
PushRecording Choice 0.6330 ns 1.000 ns
Tests DK Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.786 ns 1 ns
Control PushRecording 0.455 ns 1 ns
PushDictation PushRecording 0.771 ns 1 ns
Control Choice 0.211 ns 1 ns
PushDictation Choice 0.442 ns 1 ns
PushRecording Choice 0.785 ns 1 ns
Tests depends Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.581 ns 1 ns
Control PushRecording 0.312 ns 1 ns
PushDictation PushRecording 1.000 ns 1 ns
Control Choice 1.000 ns 1 ns
PushDictation Choice 0.749 ns 1 ns
PushRecording Choice 0.429 ns 1 ns
Tests depends Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.306 ns 0.833 ns
Control PushRecording 0.315 ns 0.833 ns
Control Choice 0.278 ns 0.833 ns
Tests depends Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.469 ns 1 ns
Control PushRecording 0.486 ns 1 ns
PushDictation PushRecording 1.000 ns 1 ns
Control Choice 0.759 ns 1 ns
PushDictation Choice 1.000 ns 1 ns
PushRecording Choice 1.000 ns 1 ns
Tests refer previous answers Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1 ns
Control PushRecording 0.852 ns 1 ns
PushDictation PushRecording 0.703 ns 1 ns
Control Choice 1.000 ns 1 ns
PushDictation Choice 1.000 ns 1 ns
PushRecording Choice 0.771 ns 1 ns
Tests refer previous answers Coder 2: NA (no “previous answer” codes).
Note that this code was added when Coder 3 revised the coding of Coders 1 and 2.


Tests refer previous answers Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1 ns
Control PushRecording 0.852 ns 1 ns
PushDictation PushRecording 0.703 ns 1 ns
Control Choice 1.000 ns 1 ns
PushDictation Choice 1.000 ns 1 ns
PushRecording Choice 0.771 ns 1 ns
Tests valid Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.599 ns 1 ns
Control PushRecording 0.499 ns 1 ns
PushDictation PushRecording 0.984 ns 1 ns
Control Choice 0.700 ns 1 ns
PushDictation Choice 0.980 ns 1 ns
PushRecording Choice 0.860 ns 1 ns
Tests valid Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.493 ns 1 ns
Control PushRecording 0.787 ns 1 ns
PushDictation PushRecording 0.808 ns 1 ns
Control Choice 0.511 ns 1 ns
PushDictation Choice 1.000 ns 1 ns
PushRecording Choice 0.836 ns 1 ns
Tests valid Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.582 ns 1 ns
Control PushRecording 0.570 ns 1 ns
PushDictation PushRecording 1.000 ns 1 ns
Control Choice 0.361 ns 1 ns
PushDictation Choice 0.836 ns 1 ns
PushRecording Choice 0.856 ns 1 ns


Main results Valid by subgroups (text versus voice)

For the Choice, PushDictation, and PushRecording groups, we also examine the proportion of valid responses (in %) by input method: text versus voice. Participants who used both input methods are classified as voice respondents. In the Choice group, the voice subgroup includes both dictation and voice recordings.

Valid WHYTRANSP C3

SUBGROUP_TRANSP valid_WHYTRANSPC3 n n_cum f f_cum
Choice_text 0 16 16 9.3 9.3
Choice_text 1 156 172 90.7 100.0
Choice_voice 0 5 5 11.1 11.1
Choice_voice 1 40 45 88.9 100.0
PushDictation_dictated 0 27 27 15.3 15.3
PushDictation_dictated 1 150 177 84.7 100.0
PushDictation_text 0 6 6 14.0 14.0
PushDictation_text 1 37 43 86.0 100.0
PushRecording_text 0 10 10 13.3 13.3
PushRecording_text 1 65 75 86.7 100.0
Pushrecording_recorded 0 11 11 8.7 8.7
Pushrecording_recorded 1 115 126 91.3 100.0


Valid WHYTRUST C3

SUBGROUP_TRUST valid_WHYTRUSTC3 n n_cum f f_cum
Choice_text 0 27 27 14.6 14.6
Choice_text 1 158 185 85.4 100.0
Choice_voice 0 8 8 20.0 20.0
Choice_voice 1 32 40 80.0 100.0
PushDictation_dictated 0 25 25 15.0 15.0
PushDictation_dictated 1 142 167 85.0 100.0
PushDictation_text 0 10 10 23.8 23.8
PushDictation_text 1 32 42 76.2 100.0
PushRecording_text 0 9 9 12.0 12.0
PushRecording_text 1 66 75 88.0 100.0
Pushrecording_recorded 0 25 25 19.4 19.4
Pushrecording_recorded 1 104 129 80.6 100.0



For valid answers only from now on


Aspects answered

Aspects WHYTRANSP

Proportions per group WHYTRANSP

Answered to both parts of the question, or only part 1 or only part 2, using the codes of Coder 1

Coder 1 Control Choice PushDictation PushRecording
Both parts 20.0 16.8 19.8 19.4
First part 57.0 60.2 52.9 50.6
Second part 23.0 23.0 27.3 30.0
Number of observations 200 196 187 180
Answered to both parts of the question, or only part 1 or only part 2, using the codes of Coder 2

Coder 2 Control Choice PushDictation PushRecording
Both parts 29.0 25.5 28.3 30.0
First part 53.0 54.1 48.1 46.1
Second part 18.0 20.4 23.5 23.9
Number of observations 200 196 187 180

Tests of significance across groups WHYTRANSP

Tests both parts Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1 ns
Control PushRecording 0.995 ns 1 ns
PushDictation PushRecording 1.000 ns 1 ns
Control Choice 0.495 ns 1 ns
PushDictation Choice 0.539 ns 1 ns
PushRecording Choice 0.602 ns 1 ns
Tests both parts Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.976 ns 1 ns
Control PushRecording 0.920 ns 1 ns
PushDictation PushRecording 0.815 ns 1 ns
Control Choice 0.505 ns 1 ns
PushDictation Choice 0.610 ns 1 ns
PushRecording Choice 0.392 ns 1 ns
Tests first part Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.4840 ns 1.000 ns
Control PushRecording 0.2480 ns 0.992 ns
PushDictation PushRecording 0.7240 ns 1.000 ns
Control Choice 0.5860 ns 1.000 ns
PushDictation Choice 0.1830 ns 0.917 ns
PushRecording Choice 0.0756 ns 0.453 ns
Tests first part Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.392 ns 1.000 ns
Control PushRecording 0.216 ns 1.000 ns
PushDictation PushRecording 0.778 ns 1.000 ns
Control Choice 0.908 ns 1.000 ns
PushDictation Choice 0.288 ns 1.000 ns
PushRecording Choice 0.150 ns 0.898 ns
Tests second part Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.394 ns 1.000 ns
Control PushRecording 0.153 ns 0.914 ns
PushDictation PushRecording 0.644 ns 1.000 ns
Control Choice 1.000 ns 1.000 ns
PushDictation Choice 0.392 ns 1.000 ns
PushRecording Choice 0.152 ns 0.914 ns
Tests second part Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.224 ns 1 ns
Control PushRecording 0.198 ns 1 ns
PushDictation PushRecording 1.000 ns 1 ns
Control Choice 0.631 ns 1 ns
PushDictation Choice 0.539 ns 1 ns
PushRecording Choice 0.491 ns 1 ns



Aspects WHYTRUST

Proportions per group WHYTRUST

Answered to both parts of the question, or only part 1 or only part 2, using the codes of Coder 1

Coder 1 Control Choice PushDictation PushRecording
Both parts 13.6 9.5 14.9 18.2
First part 67.2 75.8 65.5 59.4
Second part 19.2 14.7 19.5 22.4
Number of observations 198 190 174 170
Answered to both parts of the question, or only part 1 or only part 2, using the codes of Coder 2

Coder 2 Control Choice PushDictation PushRecording
Both parts 16.2 12.1 21.3 25.3
First part 68.7 76.8 66.7 59.4
Second part 15.2 11.1 12.1 15.3
Number of observations 198 190 174 170

Tests of significance across groups WHYTRUST

Tests both parts Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.8330 ns 1.000 ns
Control PushRecording 0.2880 ns 1.000 ns
PushDictation PushRecording 0.4990 ns 1.000 ns
Control Choice 0.2620 ns 1.000 ns
PushDictation Choice 0.1500 ns 0.752 ns
PushRecording Choice 0.0234
0.141 ns
Tests both parts Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.25900 ns 0.7760 ns
Control PushRecording 0.04150
0.1660 ns
PushDictation PushRecording 0.44900 ns 0.7760 ns
Control Choice 0.31800 ns 0.7760 ns
PushDictation Choice 0.02700
0.1350 ns
PushRecording Choice 0.00199 ** 0.0119
Tests first part Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.82000 ns 0.82000 ns
Control PushRecording 0.15200 ns 0.45500 ns
PushDictation PushRecording 0.29000 ns 0.58000 ns
Control Choice 0.07750 ns 0.31000 ns
PushDictation Choice 0.04140
0.20700 ns
PushRecording Choice 0.00131 ** 0.00786 **
Tests first part Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.761000 ns 0.76100 ns
Control PushRecording 0.081300 ns 0.32500 ns
PushDictation PushRecording 0.200000 ns 0.39900 ns
Control Choice 0.091400 ns 0.32500 ns
PushDictation Choice 0.041100
0.20600 ns
PushRecording Choice 0.000573 *** 0.00344 **
Tests second part Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1.000 ns
Control PushRecording 0.537 ns 1.000 ns
PushDictation PushRecording 0.611 ns 1.000 ns
Control Choice 0.302 ns 1.000 ns
PushDictation Choice 0.281 ns 1.000 ns
PushRecording Choice 0.084 ns 0.504 ns
Tests second part Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.477 ns 1 ns
Control PushRecording 1.000 ns 1 ns
PushDictation PushRecording 0.475 ns 1 ns
Control Choice 0.296 ns 1 ns
PushDictation Choice 0.889 ns 1 ns
PushRecording Choice 0.300 ns 1 ns


Main results Aspects answered by subgroups (text versus voice)


Both aspects answered WHYTRANSP C1

SUBGROUP_TRANSP bothparts_TRANSPC1 n n_cum f f_cum
Choice_text 0 128 128 82.1 82.1
Choice_text 1 28 156 17.9 100.0
Choice_voice 0 35 35 87.5 87.5
Choice_voice 1 5 40 12.5 100.0
PushDictation_dictated 0 122 122 81.3 81.3
PushDictation_dictated 1 28 150 18.7 100.0
PushDictation_text 0 28 28 75.7 75.7
PushDictation_text 1 9 37 24.3 100.0
PushRecording_text 0 56 56 86.2 86.2
PushRecording_text 1 9 65 13.8 100.0
Pushrecording_recorded 0 89 89 77.4 77.4
Pushrecording_recorded 1 26 115 22.6 100.0


Both aspects answered WHYTRUST C1

SUBGROUP_TRUST bothparts_TRUSTC1 n n_cum f f_cum
Choice_text 0 143 143 90.5 90.5
Choice_text 1 15 158 9.5 100.0
Choice_voice 0 29 29 90.6 90.6
Choice_voice 1 3 32 9.4 100.0
PushDictation_dictated 0 120 120 84.5 84.5
PushDictation_dictated 1 22 142 15.5 100.0
PushDictation_text 0 28 28 87.5 87.5
PushDictation_text 1 4 32 12.5 100.0
PushRecording_text 0 60 60 90.9 90.9
PushRecording_text 1 6 66 9.1 100.0
Pushrecording_recorded 0 79 79 76.0 76.0
Pushrecording_recorded 1 25 104 24.0 100.0


Abbreviations

Abbreviations WHYTRANSP

Proportions per group WHYTRANSP

Abbreviations, using the codes of Coder 1

Coder 1 Control Choice PushDictation PushRecording
Abbreviation(s) 1.5 0.5 3.7 1.7
None 98.5 99.5 96.3 98.3
Number of observations 200 196 187 180
Abbreviations, using the codes of Coder 2

Coder 2 Control Choice PushDictation PushRecording
Abbreviation(s) 1.0 1.0 4.8 1.7
None 99.0 99.0 95.2 98.3
Number of observations 200 196 187 180

Tests of significance across groups WHYTRANSP

Tests abbreviations Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.2850 ns 1.000 ns
Control PushRecording 1.0000 ns 1.000 ns
PushDictation PushRecording 0.3680 ns 1.000 ns
Control Choice 0.6300 ns 1.000 ns
PushDictation Choice 0.0637 ns 0.382 ns
PushRecording Choice 0.5560 ns 1.000 ns
Tests abbreviations Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.0512 ns 0.307 ns
Control PushRecording 0.9060 ns 1.000 ns
PushDictation PushRecording 0.1610 ns 0.645 ns
Control Choice 1.0000 ns 1.000 ns
PushDictation Choice 0.0555 ns 0.307 ns
PushRecording Choice 0.9240 ns 1.000 ns



Abbreviations WHYTRUST

Proportions per group WHYTRUST

Abbreviations, using the codes of Coder 1

Coder 1 Control Choice PushDictation PushRecording
Abbreviation(s) 3.0 1.1 3.4 0.6
None 97.0 98.9 96.6 99.4
Number of observations 198 190 174 170
Abbreviations, using the codes of Coder 2

Coder 2 Control Choice PushDictation PushRecording
Abbreviation(s) 2.0 0.5 1.1 1.2
None 98.0 99.5 98.9 98.8
Number of observations 198 190 174 170

Tests of significance across groups WHYTRUST

Tests abbreviations Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1.000 ns
Control PushRecording 0.184 ns 0.922 ns
PushDictation PushRecording 0.135 ns 0.807 ns
Control Choice 0.311 ns 0.933 ns
PushDictation Choice 0.230 ns 0.922 ns
PushRecording Choice 1.000 ns 1.000 ns
Tests abbreviations Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.800 ns 1 ns
Control PushRecording 0.822 ns 1 ns
PushDictation PushRecording 1.000 ns 1 ns
Control Choice 0.393 ns 1 ns
PushDictation Choice 0.939 ns 1 ns
PushRecording Choice 0.923 ns 1 ns



Problems detected

Problems WHYTRANSP

Proportions per group WHYTRANSP

Problems, using the codes of Coder 1

Coder 1 Control Choice PushDictation PushRecording
Incomplete sentence(s) 1.0 1.5 1.1 5.0
No problem detected 98.5 94.9 89.8 79.4
Other problem(s) NA 2.0 1.6 2.8
Part of the answer repeated 0.5 1.5 7.5 12.8
Number of observations 200 196 187 180
Problems, using the codes of Coder 2

Coder 2 Control Choice PushDictation PushRecording
Incomplete sentence(s) 1.0 1.5 1.1 4.4
No problem detected 98.5 94.9 89.3 80.0
Other problem(s) NA 2.0 2.1 3.3
Part of the answer repeated 0.5 1.5 7.5 12.2
Number of observations 200 196 187 180

Tests of significance across groups WHYTRANSP

Tests problems Coder 1: no problem detected
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 5.46e-04 *** 2.18e-03 **
Control PushRecording 0.00e+00 *** 0.00e+00 ****
PushDictation PushRecording 8.71e-03 ** 2.61e-02
Control Choice 8.38e-02 ns 1.68e-01 ns
PushDictation Choice 9.35e-02 ns 1.68e-01 ns
PushRecording Choice 1.24e-05 *** 6.21e-05 ****
Tests problems Coder 2: no problem detected
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 3.08e-04 *** 0.001230 **
Control PushRecording 0.00e+00 *** 0.000000 ****
PushDictation PushRecording 1.97e-02
0.059000 ns
Control Choice 8.38e-02 ns 0.130000 ns
PushDictation Choice 6.49e-02 ns 0.130000 ns
PushRecording Choice 2.17e-05 *** 0.000109 ***
Tests problems Coder 1: Incomplete sentence(s)
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.0000 ns 1.000 ns
Control PushRecording 0.0438
0.263 ns
PushDictation PushRecording 0.0573 ns 0.286 ns
Control Choice 0.9820 ns 1.000 ns
PushDictation Choice 1.0000 ns 1.000 ns
PushRecording Choice 0.1060 ns 0.422 ns
Tests problems Coder 2: Incomplete sentence(s)
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.0000 ns 1.000 ns
Control PushRecording 0.0761 ns 0.457 ns
PushDictation PushRecording 0.0960 ns 0.480 ns
Control Choice 0.9820 ns 1.000 ns
PushDictation Choice 1.0000 ns 1.000 ns
PushRecording Choice 0.1710 ns 0.685 ns
Tests problems Coder 1: Part of the answer repeated
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 9.85e-04 *** 3.94e-03 **
Control PushRecording 2.60e-06 *** 1.55e-05 ****
PushDictation PushRecording 1.31e-01 ns 2.62e-01 ns
Control Choice 6.01e-01 ns 6.01e-01 ns
PushDictation Choice 9.86e-03 ** 2.96e-02
PushRecording Choice 4.30e-05 *** 2.15e-04 ***
Tests problems Coder 2: Part of the answer repeated
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 9.85e-04 *** 3.94e-03 **
Control PushRecording 4.90e-06 *** 2.94e-05 ****
PushDictation PushRecording 1.77e-01 ns 3.55e-01 ns
Control Choice 6.01e-01 ns 6.01e-01 ns
PushDictation Choice 9.86e-03 ** 2.96e-02
PushRecording Choice 7.82e-05 *** 3.91e-04 ***
Tests problems Coder 1: Other problem(s)
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.2230 ns 0.892 ns
Control PushRecording 0.0546 ns 0.328 ns
PushDictation PushRecording 0.6800 ns 1.000 ns
Control Choice 0.1270 ns 0.633 ns
PushDictation Choice 1.0000 ns 1.000 ns
PushRecording Choice 0.8970 ns 1.000 ns
Tests problems Coder 2: Other problem(s)
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.1150 ns 0.575 ns
Control PushRecording 0.0285
0.171 ns
PushDictation PushRecording 0.7030 ns 1.000 ns
Control Choice 0.1270 ns 0.575 ns
PushDictation Choice 1.0000 ns 1.000 ns
PushRecording Choice 0.6470 ns 1.000 ns



Problems WHYTRUST

Proportions per group WHYTRUST

Problems, using the codes of Coder 1

Coder 1 Control Choice PushDictation PushRecording
Incomplete sentence(s) 1.0 1.6 1.1 5.9
No problem detected 97.5 95.8 90.2 74.1
Other problem(s) 1.0 NA 3.4 6.5
Part of the answer repeated 0.5 2.6 5.2 13.5
Number of observations 198 190 174 170
Problems, using the codes of Coder 2

Coder 2 Control Choice PushDictation PushRecording
Incomplete sentence(s) 1.0 1.6 4.0 2.9
No problem detected 98.0 96.8 90.8 80.6
Other problem(s) NA 0.5 1.1 4.7
Part of the answer repeated 1.0 1.1 4.0 11.8
Number of observations 198 190 174 170

Tests of significance across groups WHYTRUST

Tests problems Coder 1: no problem detected
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.006230 ** 1.87e-02
Control PushRecording 0.000000 *** 0.00e+00 ****
PushDictation PushRecording 0.000163 *** 6.51e-04 ***
Control Choice 0.522000 ns 5.22e-01 ns
PushDictation Choice 0.059100 ns 1.18e-01 ns
PushRecording Choice 0.000000 *** 1.00e-07 ****
Tests problems Coder 2: no problem detected
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 4.64e-03 ** 1.86e-02
Control PushRecording 1.00e-07 *** 5.00e-07 ****
PushDictation PushRecording 1.06e-02
3.17e-02
Control Choice 6.99e-01 ns 6.99e-01 ns
PushDictation Choice 2.82e-02
5.64e-02 ns
PushRecording Choice 1.70e-06 *** 8.60e-06 ****
Tests problems Coder 1: Incomplete sentence(s)
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.0000 ns 1.000 ns
Control PushRecording 0.0198
0.119 ns
PushDictation PushRecording 0.0359
0.179 ns
Control Choice 0.9630 ns 1.000 ns
PushDictation Choice 1.0000 ns 1.000 ns
PushRecording Choice 0.0572 ns 0.229 ns
Tests problems Coder 2: Incomplete sentence(s)
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.121 ns 0.728 ns
Control PushRecording 0.332 ns 1.000 ns
PushDictation PushRecording 0.800 ns 1.000 ns
Control Choice 0.963 ns 1.000 ns
PushDictation Choice 0.270 ns 1.000 ns
PushRecording Choice 0.605 ns 1.000 ns
Tests problems Coder 1: Part of the answer repeated
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.41e-02
5.22e-02 ns
Control PushRecording 1.30e-06 *** 8.10e-06 ****
PushDictation PushRecording 1.31e-02
5.22e-02 ns
Control Choice 1.99e-01 ns 3.97e-01 ns
PushDictation Choice 3.24e-01 ns 3.97e-01 ns
PushRecording Choice 2.55e-04 *** 1.27e-03 **
Tests problems Coder 2: Part of the answer repeated
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.21e-01 ns 0.364000 ns
Control PushRecording 3.82e-05 *** 0.000229 ***
PushDictation PushRecording 1.36e-02
0.054200 ns
Control Choice 1.00e+00 ns 1.000000 ns
PushDictation Choice 1.38e-01 ns 0.364000 ns
PushRecording Choice 5.93e-05 *** 0.000296 ***
Tests problems Coder 1: Other problem(s)
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.20800 ns 0.62400 ns
Control PushRecording 0.01090
0.05450 ns
PushDictation PushRecording 0.29600 ns 0.62400 ns
Control Choice 0.49700 ns 0.62400 ns
PushDictation Choice 0.03010
0.12000 ns
PushRecording Choice 0.00114 ** 0.00682 **
Tests problems Coder 2: Other problem(s)
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.42200 ns 1.0000 ns
Control PushRecording 0.00638 ** 0.0383
PushDictation PushRecording 0.10100 ns 0.4020 ns
Control Choice 0.98400 ns 1.0000 ns
PushDictation Choice 0.93900 ns 1.0000 ns
PushRecording Choice 0.02800
0.1400 ns



Main results Problems detected by subgroups (text versus voice)


No problem detected WHYTRANSP C1

SUBGROUP_TRANSP noproblem_TRANSPC1 n n_cum f f_cum
Choice_text 0 3 3 1.9 1.9
Choice_text 1 153 156 98.1 100.0
Choice_voice 0 7 7 17.5 17.5
Choice_voice 1 33 40 82.5 100.0
PushDictation_dictated 0 17 17 11.3 11.3
PushDictation_dictated 1 133 150 88.7 100.0
PushDictation_text 0 2 2 5.4 5.4
PushDictation_text 1 35 37 94.6 100.0
PushRecording_text 0 1 1 1.5 1.5
PushRecording_text 1 64 65 98.5 100.0
Pushrecording_recorded 0 36 36 31.3 31.3
Pushrecording_recorded 1 79 115 68.7 100.0


No problem detected WHYTRUST C1

SUBGROUP_TRUST noproblem_TRUSTC1 n n_cum f f_cum
Choice_text 0 1 1 0.6 0.6
Choice_text 1 157 158 99.4 100.0
Choice_voice 0 7 7 21.9 21.9
Choice_voice 1 25 32 78.1 100.0
PushDictation_dictated 0 16 16 11.3 11.3
PushDictation_dictated 1 126 142 88.7 100.0
PushDictation_text 0 1 1 3.1 3.1
PushDictation_text 1 31 32 96.9 100.0
PushRecording_text 0 2 2 3.0 3.0
PushRecording_text 1 64 66 97.0 100.0
Pushrecording_recorded 0 42 42 40.4 40.4
Pushrecording_recorded 1 62 104 59.6 100.0


Number of themes

No. themes WHYTRANSP

Average/Median number of themes WHYTRANSP

Number of themes, using the codes of Coder 1: average and median

Coder 1 Control Choice PushDictation PushRecording
mean 1.5 1.62 1.6 1.74
median 1.0 1.00 1.0 1.00
Number of themes, using the codes of Coder 2: average and median

Coder 2 Control Choice PushDictation PushRecording
mean 1.87 1.9 1.94 2.05
median 2.00 2.0 2.00 2.00

Tests of significance across groups WHYTRANSP (pairwise T-tests)

Tests differences average number of themes Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.21700 ns 0.5670 ns
Control PushRecording 0.00424 ** 0.0255
PushDictation PushRecording 0.10600 ns 0.5290 ns
Control Choice 0.14200 ns 0.5670 ns
PushDictation Choice 0.82800 ns 0.8280 ns
PushRecording Choice 0.15500 ns 0.5670 ns
Tests differences average number of themes Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.528 ns 1.000 ns
Control PushRecording 0.115 ns 0.687 ns
PushDictation PushRecording 0.348 ns 1.000 ns
Control Choice 0.802 ns 1.000 ns
PushDictation Choice 0.703 ns 1.000 ns
PushRecording Choice 0.185 ns 0.923 ns



No. themes WHYTRUST

Average/Median number of themes WHYTRUST

Number of themes, using the codes of Coder 1: average and median

Coder 1 Control Choice PushDictation PushRecording
mean 2.21 2.43 2.49 2.78
median 2.00 2.00 2.00 2.00
Number of themes, using the codes of Coder 2: average and median

Coder 2 Control Choice PushDictation PushRecording
mean 2.09 2.26 2.4 2.72
median 2.00 2.00 2.0 2.00

Tests of significance across groups WHYTRUST (pairwise T-tests)

Tests differences average number of themes Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 2.13e-02
8.23e-02 ns
Control PushRecording 3.30e-06 **** 1.99e-05 ****
PushDictation PushRecording 2.06e-02
8.23e-02 ns
Control Choice 6.01e-02 ns 1.20e-01 ns
PushDictation Choice 6.44e-01 ns 6.44e-01 ns
PushRecording Choice 4.78e-03 ** 2.39e-02
Tests differences average number of themes Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 2.45e-02
9.78e-02 ns
Control PushRecording 7.40e-06 **** 4.43e-05 ****
PushDictation PushRecording 2.77e-02
9.78e-02 ns
Control Choice 1.97e-01 ns 3.93e-01 ns
PushDictation Choice 3.27e-01 ns 3.93e-01 ns
PushRecording Choice 1.30e-03 ** 6.50e-03 **



Main results No. themes by subgroups (text versus voice)


No. of themes WHYTRANSP C1

SUBGROUP_TRANSP mean stdev min max median
Choice_text 1.6 0.88 1 6 1
Choice_voice 1.6 0.75 1 4 1
PushDictation_dictated 1.6 0.84 1 5 1
PushDictation_text 1.7 0.77 1 3 2
PushRecording_text 1.5 0.75 1 4 1
Pushrecording_recorded 1.9 1.01 1 6 2


No. of themes WHYTRUST C1

SUBGROUP_TRUST mean stdev min max median
Choice_text 2.4 1.07 1 6 2
Choice_voice 2.8 1.36 1 7 3
PushDictation_dictated 2.5 1.09 1 5 2
PushDictation_text 2.6 1.43 1 8 2
PushRecording_text 2.4 1.26 1 7 2
Pushrecording_recorded 3.0 1.32 1 8 3


Number of characters


Note: This is computed in R directly, no coders involved.

No. characters WHYTRANSP

Average/Median number of characters WHYTRANSP

Number of characters: average and median

No. Char Control Choice PushDictation PushRecording
mean 109 177 181 227
median 81 127 130 174

Tests of significance across groups WHYTRANSP (pairwise T-tests)

Tests differences average number of characters
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 2.10e-06 **** 1.04e-05 ****
Control PushRecording 0.00e+00 **** 0.00e+00 ****
PushDictation PushRecording 4.01e-03 ** 8.01e-03 **
Control Choice 7.30e-06 **** 2.93e-05 ****
PushDictation Choice 8.08e-01 ns 8.08e-01 ns
PushRecording Choice 1.91e-03 ** 5.73e-03 **


No. characters WHYTRUST

Average/Median number of characters WHYTRUST

Number of characters: average and median

No. Char Control Choice PushDictation PushRecording
mean 93 135 144 227
median 73 95 113 154

Tests of significance across groups WHYTRUST (pairwise T-tests)

Tests differences average number of characters
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 8.65e-04 *** 2.59e-03 **
Control PushRecording 0.00e+00 **** 0.00e+00 ****
PushDictation PushRecording 2.00e-07 **** 9.00e-07 ****
Control Choice 5.78e-03 ** 1.16e-02
PushDictation Choice 5.37e-01 ns 5.37e-01 ns
PushRecording Choice 0.00e+00 **** 0.00e+00 ****


Main results No. characters by subgroups (text versus voice)


No. of characters WHYTRANSP C1

SUBGROUP_TRANSP mean stdev min max median
Choice_text 169 140 2 1000 126
Choice_voice 210 276 4 1538 130
PushDictation_dictated 187 185 8 1423 132
PushDictation_text 160 120 6 470 122
PushRecording_text 124 111 4 491 87
Pushrecording_recorded 289 193 15 959 260


No. of characters WHYTRUST C1

SUBGROUP_TRUST mean stdev min max median
Choice_text 121 114 2 787 92
Choice_voice 196 210 6 1088 126
PushDictation_dictated 147 110 8 530 117
PushDictation_text 132 127 6 621 98
PushRecording_text 116 102 3 511 88
Pushrecording_recorded 291 302 5 2753 224


Completion time


Note: This is computed in R directly, no coders involved. All times are focus time (in seconds), i.e., times that the active page is the survey question page. Note 2: We focus only on valid answers. Note 3: For the push groups, the times reported include the potential follow-up questions, in case of skipping initially the experimental questions. Note 4: This cannot be considered a direct indicator of quality, since shorter times can be simply due to the fact that speaking is quicker than typing.

Focus time WHYTRANSP

Average/Median focus time WHYTRANSP

Focus time: average and median

Focus time Control Choice PushDictation PushRecording
mean 86 40 56 43
median 61 31 31 27

Tests of significance across groups WHYTRANSP (pairwise T-tests)

Tests differences average focus time
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 9.00e-03 ** 0.036000
Control PushRecording 2.68e-04 *** 0.001340 **
PushDictation PushRecording 2.93e-01 ns 0.586000 ns
Control Choice 5.46e-05 **** 0.000328 ***
PushDictation Choice 1.67e-01 ns 0.500000 ns
PushRecording Choice 7.59e-01 ns 0.759000 ns


Focus time WHYTRUST

Average/Median focus time WHYTRUST

Focus time: average and median

Focus time Control Choice PushDictation PushRecording
mean 68 16 16 26
median 48 8 9 9

Tests of significance across groups WHYTRUST (pairwise T-tests)

Tests differences average focus time
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.0000 **** 0.000 ****
Control PushRecording 0.0000 **** 0.000 ****
PushDictation PushRecording 0.1330 ns 0.284 ns
Control Choice 0.0000 **** 0.000 ****
PushDictation Choice 0.8910 ns 0.891 ns
PushRecording Choice 0.0945 ns 0.284 ns